Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(job-runner): surface errors to Snuba admin #6704

Merged
merged 5 commits into from
Jan 6, 2025
Merged

feat(job-runner): surface errors to Snuba admin #6704

merged 5 commits into from
Jan 6, 2025

Conversation

xurui-c
Copy link
Member

@xurui-c xurui-c commented Dec 30, 2024 •
edited
Loading

Currently, errors are surfaced to GCP, and Snuba Admin just displays server error. I want to surface the actual error message to Snuba Admin as well

@sentry-io Sentry.io
Copy link

sentry-io bot commented Dec 30, 2024

🔍 Existing Issues For Review

Your pull request is modifying functions with the following pre-existing issues:

📄 File: snuba/manual_jobs/runner.py

Function Unhandled Issue
run_job JobLockedException: Job inc-984.scrub-spans-user.whole-timerange.12_19_2024 lock exists, not available to run ...
Event Count: 1
run_job JobStatusException: Job inc-984.scrub-eap_spans-user.historical.12_20_2024 has run before, status = running, not avai... ...
Event Count: 1

Did you find this useful? React with a 👍 or 👎

@xurui-c xurui-c changed the title c feat(job-runner): surface errors to Snuba admin Dec 30, 2024
@codecov Codecov
Copy link

codecov bot commented Jan 4, 2025 •
edited
Loading

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
1163 1 1162 3
View the top 1 failed tests by shortest run time 
tests.manual_jobs.test_job_statuses::test_job_with_exception_causes_failure
Stack Traces | 0.029s run time 
Traceback (most recent call last):
  File ".../tests/manual_jobs/test_job_statuses.py", line 48, in test_job_with_exception_causes_failure
    run_job(test_job_spec)
  File ".../snuba/manual_jobs/runner.py", line 141, in run_job
    raise e
  File ".../snuba/manual_jobs/runner.py", line 133, in run_job
    job_to_run.execute(job_logger)
  File ".../tests/manual_jobs/test_job_statuses.py", line 22, in execute
    raise SerializableException("Intended failure")
snuba.utils.serializable_exception.SerializableException: Intended failure

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

@xurui-c xurui-c marked this pull request as ready for review January 4, 2025 00:26
@xurui-c xurui-c requested a review from a team as a code owner January 4, 2025 00:26
onkar
onkar approved these changes Jan 6, 2025
View reviewed changes
snuba/manual_jobs/runner.py Outdated
@@ -134,10 +134,11 @@ def run_job(job_spec: JobSpec) -> JobStatus:
if not job_spec.is_async:
current_job_status = _set_job_status(job_spec.job_id, JobStatus.FINISHED)
job_logger.info("[runner] job execution finished")
except BaseException as e:
except Exception as e:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did this exception type need change? AFAIK, BaseException is used to catch all exceptions including the ones needed to terminate the program, while Exception is used to catch exceptions that do not terminate the program.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My mistake, fixed!

@xurui-c xurui-c enabled auto-merge (squash) January 6, 2025 19:07
@xurui-c xurui-c merged commit af48265 into master Jan 6, 2025
31 checks passed
@xurui-c xurui-c deleted the rachel/joberror branch January 6, 2025 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@onkar onkar onkar approved these changes

@evanh evanh evanh approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@xurui-c @onkar @evanh